SHADHO: Massively Scalable Hardware-Aware Distributed Hyperparameter Optimization

Computer vision is experiencing an AI renaissance, in which machine learning models are expediting important breakthroughs in academic research and commercial applications. Effectively training these models, however, is not trivial due in part to hyperparameters: user-configured values that control a model's ability to learn from data. Existing hyperparameter optimization methods are highly parallel but make no effort to balance the search across heterogeneous hardware or to prioritize searching high-impact spaces. In this paper, we introduce a framework for massively Scalable Hardware-Aware Distributed Hyperparameter Optimization (SHADHO). Our framework calculates the relative complexity of each search space and monitors performance on the learning task over all trials. These metrics are then used as heuristics to assign hyperparameters to distributed workers based on their hardware. We first demonstrate that our framework achieves double the throughput of a standard distributed hyperparameter optimization framework by optimizing SVM for MNIST using 150 distributed workers. We then conduct model search with SHADHO over the course of one week using 74 GPUs across two compute clusters to optimize U-Net for a cell segmentation task, discovering 515 models that achieve a lower validation loss than standard U-Net.Comment: 10 pages, 6 figure

LSST: from Science Drivers to Reference Design and Anticipated Data Products

(Abridged) We describe here the most ambitious survey currently planned in the optical, the Large Synoptic Survey Telescope (LSST). A vast array of science will be enabled by a single wide-deep-fast sky survey, and LSST will have unique survey capability in the faint time domain. The LSST design is driven by four main science themes: probing dark energy and dark matter, taking an inventory of the Solar System, exploring the transient optical sky, and mapping the Milky Way. LSST will be a wide-field ground-based system sited at Cerro Pach\'{o}n in northern Chile. The telescope will have an 8.4 m (6.5 m effective) primary mirror, a 9.6 deg$^2$ field of view, and a 3.2 Gigapixel camera. The standard observing sequence will consist of pairs of 15-second exposures in a given field, with two such visits in each pointing in a given night. With these repeats, the LSST system is capable of imaging about 10,000 square degrees of sky in a single filter in three nights. The typical 5$\sigma$ point-source depth in a single visit in $r$ will be $\sim 24.5$ (AB). The project is in the construction phase and will begin regular survey operations by 2022. The survey area will be contained within 30,000 deg$^2$ with $\delta<+34.5^\circ$, and will be imaged multiple times in six bands, $ugrizy$, covering the wavelength range 320--1050 nm. About 90\% of the observing time will be devoted to a deep-wide-fast survey mode which will uniformly observe a 18,000 deg$^2$ region about 800 times (summed over all six bands) during the anticipated 10 years of operations, and yield a coadded map to $r\sim27.5$. The remaining 10\% of the observing time will be allocated to projects such as a Very Deep and Fast time domain survey. The goal is to make LSST data products, including a relational database of about 32 trillion observations of 40 billion objects, available to the public and scientists around the world.Comment: 57 pages, 32 color figures, version with high-resolution figures available from https://www.lsst.org/overvie

Global airborne sampling reveals a previously unobserved dimethyl sulfide oxidation mechanism in the marine atmosphere

6 pags., 5 figs.Dimethyl sulfide (DMS), emitted from the oceans, is the most abundant biological source of sulfur to the marine atmosphere. Atmospheric DMS is oxidized to condensable products that form secondary aerosols that affect Earth’s radiative balance by scattering solar radiation and serving as cloud condensation nuclei. We report the atmospheric discovery of a previously unquantified DMS oxidation product, hydroperoxymethyl thioformate (HPMTF, HOOCH2SCHO), identified through global-scale airborne observations that demonstrate it to be a major reservoir of marine sulfur. Observationally constrained model results show that more than 30% of oceanic DMS emitted to the atmosphere forms HPMTF. Coincident particle measurements suggest a strong link between HPMTF concentration and new particle formation and growth. Analyses of these observations show that HPMTF chemistry must be included in atmospheric models to improve representation of key linkages between the biogeochemistry of the ocean, marine aerosol formation and growth, and their combined effects on climate.Additional National Oceanic and Atmospheric Administration (NOAA) support for ATom was provided by NASA funding via Inter-Agency Transfer NNH15AB12l and by funding from the NOAA Climate Program Office and the NOAA Atmospheric Chemistry, Carbon Cycle, and Climate program. K.H.M. and H.G.K. acknowledge the financial support of the Independent Research Fund Denmark, the University of Copenhagen, and the Danish Ministry for Higher Education and Science’s Elite Research travel grant. J.L.J.’s group acknowledges NASA grants NHX15AH33A and 80NSSC19K0124. A.K. was supported by the Austrian Science Fund’s Erwin Schrodinger Fellowship. A.S.-L., Q.L., and C.A.C. are supported by the European Research Council (ERC) Executive Agency under the European Union’s Horizon 2020 Research and Innovation programme (Project ERC-2016-COG 726349 CLIMAHAL). The National Center for Atmospheric Research is sponsored by the National Science Foundation. E.A. and J.B. were funded in part by NASA Atmospheric Composition Program. T.H.B. and C.M.J. acknowledge support from the National Science Foundation Center for Aerosol Impacts on Chemistry of the Environment under grant CHE 1801971. B.W. and M.D. have received funding from the ERC under the European Union’s Horizon 2020 research and innovation framework program under grant 640458 (A-LIFE) and from the University of Vienna.Peer reviewe